Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets

Global power systems are increasingly reliant on wind energy as a mitigation strategy for climate change. However, the variability of wind energy causes system reliability to erode, resulting in the wind being curtailed and, ultimately, leading to substantial economic losses for wind farm owners. Wind curtailment can be reduced using battery energy storage systems (BESS) that serve as onsite backup sources. Yet, this auxiliary role may significantly hamper the BESS's capacity to generate revenues from the electricity market, particularly in conducting energy arbitrage in the Spot market and providing frequency control ancillary services (FCAS) in the FCAS markets. Ideal BESS scheduling should effectively balance the BESS's role in absorbing onsite wind curtailment and trading in the electricity market, but it is difficult in practice because of the underlying coordination complexity and the stochastic nature of energy prices and wind generation. In this study, we investigate the bidding strategy of a wind-battery system co-located and participating simultaneously in both the Spot and Regulation FCAS markets. We propose a deep reinforcement learning (DRL)-based approach that decouples the market participation of the wind-battery system into two related Markov decision processes for each facility, enabling the BESS to absorb onsite wind curtailment while simultaneously bidding in the wholesale Spot and FCAS markets to maximize overall operational revenues. Using realistic wind farm data, we validated the coordinated bidding strategy for the wind-battery system and find that our strategy generates significantly higher revenue and responds better to wind curtailment compared to an optimization-based benchmark. Our results show that joint-market bidding can significantly improve the financial performance of wind-battery systems compared to individual market participation

Optimal Energy Storage Scheduling for Wind Curtailment Reduction and Energy Arbitrage: A Deep Reinforcement Learning Approach

Wind energy has been rapidly gaining popularity as a means for combating climate change. However, the variable nature of wind generation can undermine system reliability and lead to wind curtailment, causing substantial economic losses to wind power producers. Battery energy storage systems (BESS) that serve as onsite backup sources are among the solutions to mitigate wind curtailment. However, such an auxiliary role of the BESS might severely weaken its economic viability. This paper addresses the issue by proposing joint wind curtailment reduction and energy arbitrage for the BESS. We decouple the market participation of the co-located wind-battery system and develop a joint-bidding framework for the wind farm and BESS. It is challenging to optimize the joint-bidding because of the stochasticity of energy prices and wind generation. Therefore, we leverage deep reinforcement learning to maximize the overall revenue from the spot market while unlocking the BESS's potential in concurrently reducing wind curtailment and conducting energy arbitrage. We validate the proposed strategy using realistic wind farm data and demonstrate that our joint-bidding strategy responds better to wind curtailment and generates higher revenues than the optimization-based benchmark. Our simulations also reveal that the extra wind generation used to be curtailed can be an effective power source to charge the BESS, resulting in additional financial returns.Comment: 2023 IEEE Power & Energy Society General Meeting (PESGM). arXiv admin note: text overlap with arXiv:2212.1336

Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets

Driven by the global decarbonization effort, the rapid integration of renewable energy into the conventional electricity grid presents new challenges and opportunities for the battery energy storage system (BESS) participating in the energy market. Energy arbitrage can be a significant source of revenue for the BESS due to the increasing price volatility in the spot market caused by the mismatch between renewable generation and electricity demand. In addition, the Frequency Control Ancillary Services (FCAS) markets established to stabilize the grid can offer higher returns for the BESS due to their capability to respond within milliseconds. Therefore, it is crucial for the BESS to carefully decide how much capacity to assign to each market to maximize the total profit under uncertain market conditions. This paper formulates the bidding problem of the BESS as a Markov Decision Process, which enables the BESS to participate in both the spot market and the FCAS market to maximize profit. Then, Proximal Policy Optimization, a model-free deep reinforcement learning algorithm, is employed to learn the optimal bidding strategy from the dynamic environment of the energy market under a continuous bidding scale. The proposed model is trained and validated using real-world historical data of the Australian National Electricity Market. The results demonstrate that our developed joint bidding strategy in both markets is significantly profitable compared to individual markets

Positive Solutions of Nonlocal Boundary Value Problem for High-Order Nonlinear Fractional -Difference Equations

We study the nonlinear -difference equations of fractional order ( )( ) + ( , ( )) = 0, 0 < < 1, ( where is the fractional -derivative of the Riemann-Liouville type of order , − 1 < ≤ , > 2, 1 ≤ ≤ − 2, and 0 ≤ ≤ 1. We obtain the existence and multiplicity results of positive solutions by using some fixed point theorems. Finally, we give examples to illustrate the results

Vehicle trajectory based control delay estimation at intersections using low-frequency floating car sampling data

Control delay is an important parameter that is used in the optimization of traffic signal timings and the estimation of the level of service at signalized intersection. However, it is also a parameter that is very difficult to estimate. In recent years, floating car data has emerged as an important data source for traffic state monitoring as a result of high accuracy, wide coverage and availability regardless of meteorological conditions, but has done little for control delay estimation. This article proposes a vehicle trajectory based control delay estimation method using low-frequency floating car data. Considering the sparseness and randomness of low-frequency floating car data, we use historical data to capture the deceleration and acceleration patterns. Combined with the low-frequency samples, the spatial and temporal ranges where a vehicle starts to decelerate and stop accelerating are calculated. These are used together with the control delay probability distribution function obtained based on the geometric probability model, to calculate the expected value of the control delay for each vehicle. The proposed method and a reference method are compared with the truth. The results show that the proposed method has a root mean square error of 11.8 s compared to 13.7 s for the reference method for the peak period. The corresponding values for the off-peak period are 9.3 s and 12.5 s. In addition to better accuracy, the mean and standard deviation statistics show that the proposed method outperforms the reference method and is therefore, more reliable. This successful estimation of control delay from sparse data paves the way for a more widespread use of floating car data for monitoring the state of intersections in road networks. First published online 5 February 202

Exercise in Patients with Non-cardiac Myocardial Injury under Optical Microscope

In recent years, the incidence of cardiovascular disease has been increasing year by year, has gradually developed into a global health problem, in the world wide concern. Non - cardiac injury is one of the most common cardiovascular diseases. The rehabilitation of patients with non-cardiac myocardial injury is related to their life and quality of life. Rehabilitation exercise is helpful to improve the therapeutic effect of patients. The purpose of this paper is to explore the specific effects of rehabilitation exercise on non-cardiac myocardial injury and to promote the full play of the role of rehabilitation exercise in the treatment of noncardiac myocardial injury. First to illustrate the application of optical microscope, and then from the specific reflection of non cardiac myocardial damage and formation mechanism are introduced, based on the cases were retrospectively analyzed experiment, to explore the rehabilitation exercise in the cardiac effects of myocardial injury treatment, and on the basis of this puts forward the corresponding scientific rehabilitation exercise plan. Experimental results show that compared with the rehabilitation exercise intervention before and after rehabilitation exercise intervention, non cardiac myocardial injury in the therapeutic effect of 17% or so, in the treatment of speed increased by about 21%, in the recurrence rate was reduced by 17% or so, so sports in promoting the cardiac myocardial injury treatment has good effect

Distributed gene clinical decision support system based on cloud computing

Background: The clinical decision support system can effectively break the limitations of doctors’ knowledge and reduce the possibility of misdiagnosis to enhance health care. The traditional genetic data storage and analysis methods based on stand-alone environment are hard to meet the computational requirements with the rapid genetic data growth for the limited scalability. Methods: In this paper, we propose a distributed gene clinical decision support system, which is named GCDSS. And a prototype is implemented based on cloud computing technology. At the same time, we present CloudBWA which is a novel distributed read mapping algorithm leveraging batch processing strategy to map reads on Apache Spark. Results: Experiments show that the distributed gene clinical decision support system GCDSS and the distributed read mapping algorithm CloudBWA have outstanding performance and excellent scalability. Compared with state-of-the-art distributed algorithms, CloudBWA achieves up to 2.63 times speedup over SparkBWA. Compared with stand-alone algorithms, CloudBWA with 16 cores achieves up to 11.59 times speedup over BWA-MEM with 1 core. Conclusions: GCDSS is a distributed gene clinical decision support system based on cloud computing techniques. In particular, we incorporated a distributed genetic data analysis pipeline framework in the proposed GCDSS system. To boost the data processing of GCDSS, we propose CloudBWA, which is a novel distributed read mapping algorithm to leverage batch processing technique in mapping stage using Apache Spark platform. Keywords: Clinical decision support system, Cloud computing, Spark, Alluxio, Genetic data analysis, Read mappin